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SUBSTITUT E SPECIFIC ATION 

METHOD FOR MAKING A COLORFUL 3D MODEL 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

[0001] The present invention relates to a method for constructing a three -dimensional model, 
and more particularly to a method using the transformation of a generic model with regular 
mesh structure embedded therein to have the regularly transformed mesh structure. Further, the 
method is able to automatically compensate color difference between two adjacent transformed 
mesh to reach a highly realistic surface effect. 

2. Description of Related Art 

[0002] N owadays, a computer -generated three -dimensional (3D) model has been widely used 
in different fields, i.e. from the characters in video games to special visual effect in the movie 
industry or from the commercial development of multi-media to4he special requirements in 
the medical industry. As a consequence, the construction and operation of 3D data or 3D model 
have become crucial lessons in the field of making a 3D model. 

[00031 The conventional way of making a 3D model starts from the drafting of the animation 
engineer by using the modeling software. Normally, it took a long time to train a qualified 
animation engineer. After the animation engineer is qualified, the engineer still has to use 
creativity to add in "personal touch" in the modeling process and also the coloring to the 
finished model to make the model as m«eh-perfect as possible. This art creation process takes 
a long time to finalize the entire process. Also, the "personal touch" sometimes may become 
the greatest failure in the entire creation process. 

[0004] In contrast to the conventional model creation method, using measurement devices to 
construct a 3D model belongs to the category of reverse engineering. The shape and color 
information can be retrieved by using delicate devices with tfee-O.Olcm or higher accuracy. The 
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measured data of the shape and color of an object usually is presented by a triangular mesh or 
curved surface to show the geometry information, which is shown in Fig. 1 A. A two 
-dimensional image is shown in Fig. IB to show the color information. The interrelationship 
between the color information and the geometry information is shown by texture mapping. The 
mapping is often referred i^Las texture coordinate. In order to have a complete model,j| 
different measurement angle to the object is required. Then the measured data is adjusted and 
integrated into the same spatial coordinate system, which is shown in Fig. 1C. Thereafter, the 
data is integrated into a complete 3D model as shown in Fig. ID. 

[0005] The model created by reverse engineering process has high accuracy in relation to the 
object. The difference is hardly to be recognized by thenaked ayese^g. Besides, there is no 
special training program required for the operator. The operator only needs to be familiar with 
the equipment. However, the data obtained from the measurement instrument usually is 
enormous and lack ofl acks regularity, which results in that the data can only be used in the 
production of a specific object. Besides, the large quantity of data hinders the post-process, e.g. 
data transmission or data reproduction. Furthermore, as an influence from the light, the data 
from different measuring andean gles has obvious color difference. Therefore, a complete 
method to practically use the original measured data is required to solve the previously 
described problems. 

[0006] To men dcorrect the problems, some recommends it is recommended to construct a 3D 
model by using special tools. Still the time spent for manually constructing a 3D model does 
not meet the cost-effectiveness requirement. Due to the fast growth of reverse engineering, a 
highly precise measurement instrument is applied to retrieve anobject's 3D data to recreate a 
vivid model out of the object measured. 

100071 U .S. Pat. No. 6,512,518 (the '518 patent) discusses a method of using a laser scanning 
device to retrieve object 3D data and then the obtained data is transformed into meshed data. A 
method for integrating the meshed data is also provided. The '518 patent is able to quickly and 
accurately measure the spatial position of an object so that a highly accurate model is produced. 
However, the spatial position is represented by a densed dot group data, which is large and 
irregular. Consequently, the re-use of the measured data is highly unlikely. U.S. Pat. No. 
6,356,272 ( £ 272 patent) applies shape from jLsilhouette principle to utilize aiixed camera 
system to take large am o unt numbers of pictures to create a 3D model from the continuous 



images and establish the mapping relationship between the images and the mesh. The pictures 
taken by the '272 patent are continuous from the sides of the object. The best mapping 
relationship is chosen from the angle between the normal of a triangle and the image. The top 
and bottom of an object or an object with a complex appearance may have data distortion when 
mapping occurred. 

[00081 To overcome the shortcomings, the present invention tends to provide an improved 
method to make a vivid and colorful model to mitigate the aforementioned problems. 

SUMMARY OF THE INVENTION 

[00091 The primary objective of the present invention is to provide an improved method which 
is able to integrate the retrieved data into-a complete colorful information so as to establish a 
vivid 3D model. Besides, the retrieved data from an object is mapped to a generic model having 
regular data embedded therein. After mapping, the data from the generic model is forced to 
transform into usable, regular geometry information for the model. 

[00101 Another objective of the present invention is to provide a color mending method to 
compensate for color difference between adjacent data such that the surface color on the model 
is smooth and continuous. 

[00111 Other objects, advantages and novel features of the invention will become more 
apparent from the following detailed description when taken in conjunction with the 
accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[00121 Fig. 1 A is ^schematic view to show the geometry information of a picture by using the 
triangular mesh or a curved surface; 

[00131 Fig. IB is a schematic view showing the color information of a two -dimensional 
image; 

[0014| Fig. 1C is a schematic view showing the integration of all the measured data into the 
same spatial coordinate; 

100151 Fig. ID is a schematic view showing that all the measured data is integrated into a 
complete three -dimensional model; 

[00161 F ig. 2 is a flow chart showing the production of the 3D model; 



[0017] Fig. 3 A is a schematic view showing the original mesh as measured mesh b y the 
measuring instrument; 

[00181 Fig. 3B is a schematic view of mesh of a new model with precise and regular data; 
[0019] Fig. 3C is a schematic view showing color difference between adjacent mesh; 
[00201 Fig. 4 is a flow chart of reconstructing regular mesh model; 

1 002 I I Fig. 5 is a schematic view showing the original mesh and the corresponding color 
information; 

[0022] Fig. 6 is a schematic view of the selected generic model; 

[00231 Fig. 7 is a schematic view of the transformed generic model; 

[0024] Fig. 8A is a schematic view showing the original measured data; 

(00251 Fig. 8B is a schematic view of the reconstructed model by using the generic model of 
Fig. 6; 

[00261 Fig. 9 is a schematic view showing that the texture image data is extracted from the 
original measured data; 

[0027] F ig. 10 is a flow chart of abstracting color map information; 

[0028| Fig. 11 A is a schematic view showing the spatial interrelationship between the texture 
image data of the original measured data and the generic model ; 

[00291 Fig. 1 IB is a schematic view showing that the texture image data is reattac h reajaLac^hed 
to the generic model to complete the color abstracting process; 

[00301 Fig. 12 is a flow chart showing the harmonization of color between two measured 
mesh; 

[0031 1 Fig. 13 is a schematic view of the overlapping relationship and the arrangement 
sequence of the measured data; 

[00321 Fig. 14 is a schematic view showing that the overlapped portion of two adjacent mesh 
medelmodels and the overlapped portion corresponds correspond to respective texture image; 

[00331 Figs. 15A and 15B are ^comparison between the mesh models before and after 
adjustment of color average; 

100341 F ig. 16 is a flow chart showing the pixel blending; and 



ft g[00351 Figs. 17 A, 17B and 17C are schematic views showing the advanced comparison 
result from Figs. 15Aand 15B. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

[0036] The present invention relates to a method of processing three -dimensional (3D) data to 
integrate the measured 3D data from the object to be reproduced into a complete 3D color 
model. In the geometry information aspect, the method applies a generic model to combine 
measured data from different angles of the object to become a mesh model with regular data 
embedded therein. 

[0037] In the color information aspect, the method applies the comparison between the spatial 
corresponding relationship of the newly produced regular data of the mesh model and the 
original measured data to reattach the texture image data of the measured data to the model. 
The color difference between adjacent images is then adjusted so that by means of interactive 
measurement, the operator is able to easily construct a 3D model with high accuracy and 
applicability. 

[00381 The method uses a generic model to circumstantially integrate the original measured 
data into a complete model. The word "generic" means that the model is applicable to all sorts 
of objects with the similar appearances such that severe distortion may be avoided. For 
example, to construct a human head model, a generic model with facial characteristics^ i.e. a 
pair of eyes, a nose, a mouth and a pair of ears a may be applied. To construct an animal such as 
a cow, a horse or even a sheep, a generic model with four legs may be applied. 

100391 The original and vast mesh of the object is not dealt with in the present invention but to 
adopt the pre-designed generic model with regular mesh structure to map with the original 
measured data such that a rough model with the same appearance as that of the object being 
measured is built. If there is any data breakage such as hair or other parts of the object that are 
not e asy to be easjly measured, the breakage may be mended automatically by applying the 
mesh structural relationship between adjacent data in the mapping process. The corresponding 
relationship of the texture images is automatically established by using the spatial relationship 
between the generic model and the measured data without the involvement of special 
positioning equipment or any manual operation. 

[00401 The method of the present invention mainly is divided into four major parts: 
reconstructing ^regular mesh model; 



abstracting color; 

harmonization of color arrangement; and 
pixel blending between overlapped images. 

[0041] With reference to Fig. 2, the first step of the present invention is to reconstruct ^regular 
mesh model. The data measured by the three -dimensional measuring device is dense, as shown 
in Fig. 3 A, to reduce the error by replacing the curved surface model with the mesh model. This 
is particula r aarticularly true for those objects with complicated shapes or fine minute 
characteristics. However, the more accurate the measured data is, the larger quantity of the 
triangular mesh it becomes. Therefore, direct implication of the original measured data may 
lead to a consequence that the quantity of the meshes is large and is not applicable. 

[0042] Therefore, i*sk*git is recomm ended to use a generic model with regular mesh structure 
embedded therein to map with the original measured data to generate a new model. The new 
model has a regular mesh structure inherited from the generic model; meanwhile, it is 
transformed to a shape similar to the original measured data, as shown in Fig. 3B. Further, due 
to the overlapping relationship in the spatial positions between the original measured data and 
the data of the new model, the texture images from the original measured data are projected to 
the new model so as to establish the corresponding relationship between the new model and the 
texture images. 

[0043 1 When the second step is finished, the construction of a complete model with regular 
mesh structure and multiple color texture images is completed. However, due to the color 
difference between the texture images take staken from different viewing angles, as shown in 
Fig. 3C, the overlapping areas of images isare used to adjust the color difference to allow the 
brightness of all the images b e comes to become the same. Pixel blending is processed in the 
image overlapping area so that a 3D model, as shown in Fig. 3D, with concise mesh structure 
and smooth surface color is generated. 

[00441 With reference to Fig. 4, the original measured data (100) is a group of mesh models 
obtained byj| 3D measuring device. Each model is composed of mesh data (110) and texture 
image data (120), obtained by measurement in different angles to an object to be reproduced. 
All of the models are then transformed into the same coordinate system^ In step (SI 02), 
according to the shape of the object, a generic model (200) with theg similar appearance to the 
object is selected. In step (SI 04), the generic model is roughly overlapped with the original 



measured data in space. In step (SI 06), the dimension of the generic model is adjusted to 
correspond to the dimension of the original measured data. In thej ast step (SI 08), the generic 
model is projected to the original model. Consequently, the data of the generic model is 
deformed and thus the generic model has an appearance similar to that of the original measured 
data (100). Even so, the data of the generic model still has ^regular mesh structure 
characteristic. 

Figwef 0045J Fig. 6 shows a generic model (200) ready for use in the present invention. 
Figu r eH g, 7 shows the appearance change of the deformed generic model (210). F i gureE ifc 
8A and Figur e Fig. 8B show the differences in the quantity of mesh and mesh distribution 
between the original measured data (100) (Fig. 8A) and the deformed generic model (210). 

[00461 Color abstracting is used to separate texture image data (120) from the original 
measured data (100). Then the texture image (120) is re-mapped to the deformed generic model 
(210), as shown in Fig. 9. As a matter of fact, to establish the corresponding relationship 
between the deformed generic model (210) and the texture image (120), the texture coordinate 
and the corresponding texture image of each mesh point of the deformed generic model (210) 
isare required. Because each mesh point of the deformed generic model (210) is projected to 
the original measured data (100), the triangle cont a ins contaimng the projected mesh point is 
used to calculate the texture coordinate^ and the texture image corresponding to the triangle of 
the deformed generic model mesh point is used as the corresponding texture image. 

f00471 With reference to Fig. 10, step (S202) is to choose the corresponding triangle of mesh 
point of the deformed genetic model (210). Step (S204) is to calculate the texture coordinate of 
the chosen triangle of the mesh point of the deformed generic model (210) to correspond to the 
texture image. In step (S206), checkt he continuity of the triangles chosen is checked to see if 
the chosen triangles are within the same texture image. If the coordinates of the chosen 
triangles are not continuous, not within the same texture image, other triangles should be 
selected to calculate the corresponding coordinates, which is shown in step (S208). Then to 
complete the calculation of the coordinates of all the triangles of the deformed generic model 
(210), in step (S210), a process is required to check if all the triangles of the deformed generic 
model (210) are calculated. 

[0048] With reference to Figs. 1 1 A and 1 IB, after color abstraction, the generic model (220) is 
a three -dimensional colorful model and contains multiple texture images. However, because 



the texture images are taken from different angles, color differences between the texture 
images may occur. In order to harmonize the surface color of the generic model (220), the 
overlapped characteristic between the texture images is used. 

[00491 With reference to Fig. 12, step (S302) is to seek the overlapped area Oij between 
measured data (100). That is, if the measured data (100) is M={Mi, M 2 , M 3 ,. . .M n }, n three 
-dimensional mesh, Ojj stands for the overlapped area between any two adjacent measured data 
Mi, Mj. In step (S304), the magnitude of Oij is determined. Then in step (S306), the sequence of 
M is determined. Therefore, if Mi is the first layer Mli, all the mesh model related to and 
overlapped with Mi is M L 2- Thus all the mesh model related to Mu is M L 3 and so on. Each 
mesh model in each layer is arranged in a descending manner according to their magnitudes. 
Thus a new three -dimensional mesh model group M ' ={M ' i,M ' 2,„,,M ' n } is obtained. 

FH*m= e[0050[ Fig. 13 shows the overlapping relationship and the layer sequence of the 
measured data. Figure Figu 14 shows the overlapped area between two adjacent mesh 
medel models and the overlapped areas respectively correspond to their own texture images. 

100511 In step (S308), according to M ' sequence, the color adjustment value A\ of the texture 
image of the mesh model is calculated in relation to the intensity average of the overlapped area 
of each mesh model, which is: 

Intensity average value of the overlapped area of M'i is: I A vQi =1,2,3,. . .,n 

Color adjustment value of M'i Ai=l 

[0052] Then the color adjustment value of M'i influenced by M'i is Ai,i=Aix(I AV Qi/lAVQiX 

[0053] Then if all the mesh models that are overlapped with M'i are taken into consideration, 

color adjustment value of M'i would be : 

AKA U xw u +. . ..+A i , i ,xW i ,, 1 )/(W u +. . .+W u .O 
where W* is the mesh influenced weight value. 

FfgtH=e|00541 Fie. 15 shows the comparison between and after color average adjustment of a 
group of mesh models, wherein Fig. 15A is before the color adjustment and Fig. 15B is after 
color adjustment. 

[0055] Pixel blending is processed to the images in the overlapped areas to harmonize the color 
of two adjacent images. 



[00561 With reference to Fig. 16, kestep (S402)H4 is to seek all the overlapped triangles and 
the texture images covered by the triangles. To triangle T, if the corresponding texture images 
are I t ,i,It,2. • • JT,m, the m texture images isare overlapped in all the parts of Ti,i,Ti i 2„„Ti fm in 
relation to triangle T. Therefore, pixel blending is processed to these overlapped mapped areas. 

[0057] In step (S404), to each triangle T in the overlapped area, calculation of the distances D 
of the vertices of the triangle T to the nearest boundary vertex is required. Because the triangle 
T has m corresponding mesh models, the distances Dl, D2. . .,Dm are obtained by calculating 
each vertex of the triangles. In step (S406), each triangle in the overlapped area is used as an 
unit so that pixel blending weight average is processed to the texture image corresponding area 
covered by the unit. To the vertex Vi(i=l,2,3) of each triangle, the weight of the pixel blending 
is Di,i,D i>2 ,. . ..Di, m . The pixel color of the covered images is Q,i,Ci, 2 , . - .Ci, m . The color after 
pixel blending is C iAV G. To every sampling point within the triangle, the pixel blending weight 
is calculated by applying the barycentric coordinate principle. Then the color after pixel 
blending is processed by using the same principle-; 

Q^voKCuxDu+Ci^xDi^. . .+(Ci, m xDi, m ) 
[00581 Fig. 17 is an advanced comparison to Fig. 15, wherein Fig. 17C is the result after pixel 
blending to the overlapped area of Fig. 17B. 

In reference to the [0059J The following tables ^ it is nppr o cinted to learn illustrate the 
advantages of the present invention. 
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f00601 It is to be understood, however, that even though numerous characteristics and 
advantages of the present invention have been set forth in the foregoing description, together 
with details of the structure and function of the invention, the disclosure is illustrative only, and 
changes may be made in detail, especially in matters of shape, size^ and arrangement of parts 
within the principles of the invention to the full extent indicated by the broad general meaning 
of the terms in which the appended claims are expressed. 
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